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Abstract 
In response to a request for additional analyses, in particular reporting confidence intervals 
around the results, we re-analyzed the data from prior studies. This supplementary report 
presents the results of the additional analyses addressing classification accuracy, reliability, and 
criterion-related validity evidence. For ease of reference, we organize this technical report into 


sections based on the type of evidence being presented. 


Supplementary Report on easy CBM MCRC Measures: 
A Follow-Up to Previous Technical Reports 
This technical report is an addendum to previous technical reports. In response to a 
request for additional analyses, in particular reporting confidence intervals around the results, we 
re-analyzed the data from prior studies. This supplementary report presents the results of the 
additional analyses addressing classification accuracy, reliability, and criterion-related validity 
evidence. For ease of reference, we organize this technical report into sections based on the type 
of evidence being presented. 
Classification Accuracy Methods 
We used the Smarter Balanced English Language Arts Assessment as our criterion 
measure. This measure is completely independent from the screening measure. SBAS is a large- 
scale assessment in wide use across the United States as a state accountability measure. We used 
R statistical package to perform the classification analyses. The cut point of the score associated 
with the 40" percentile from the easyCBM National Norms was selected, as prior studies and 
wide-spread district policy suggests this is an appropriate cut-point for identifying students with 
intensive need. Although the 40" percentile might, initially, seem too high a cut-point for 
intensive need, the higher expectations for student performance aligns with the higher 
expectations for which schools are being held accountable in the past five years. (Prior to SBAS 
and the CCSS adoption, performance expectations in the states from which this sample was 
drawn were substantially lower — the 20" percentile was previously used for identifying students 
with intensive need. Expectations have increased, however, and thus our cut-point also had to 


raise. 


Students who scored below the cut-point 40" percentile were assigned a variety of 
interventions, depending on specific pattern of need (performance on other parts of the literacy 
benchmark assessment such as vocabulary and reading comprehension, success of prior years’ 
interventions, whether they also had identified mathematics needs) and resources available at the 
schools. Interventions ranged from one-on-one daily instruction on phonics to small group (2-6 
students) twice-weekly supplemental fluency instruction, to after-school mentoring with a focus 
on oral reading fluency. A number of students concurrently received several of these 
interventions (typically only those students whose mathematics performance did not indicate a 
need for mathematics intervention as well because those students who also needed mathematics 
intervention simply did not have sufficient time in the school day to receive all the instructional 
interventions they needed). Interventions were delivered by a variety of personnel (depending on 
school/district resources): Special Education teachers, general education teachers during their 
“intervention block”, instructional assistants, and student mentors (some adult, some older 


children). Sample demographics are reported in Table 1. 


Table 1 
Sample Demographics, Classification Accuracy Analyses 
Grade 3 4 5 6 7 8 
Criterion SBAS ELA SBAS ELA SBAS ELA SBAS ELA SBAS ELA SBAS ELA 
Nationaal Pacific Pacific Pacific Pacific Pacific Pacific 
Representa ont Northwest, | Northwest, | Northwest, | Northwest, | Northwest, | Northwest, 
OR and WA | OR and WA | OR and WA | OR and WA | OR and WA | OR and WA 
Date SY2014-15 | SY2014-15 | SY2014-15 | SY2014-15 | SY2014-15 | SY2014-15 
Sample Size 26250 30567 30483 29800 29267 34250 
Male 12667 12100 12517 12117 11817 13783 
Female 11467 11800 11667 11417 11133 13317 
Gender Unknown 2117 6667 6300 6267 6317 7150 
Bice OL Redicrd: Duce 8133 8233 7933 8300 7433 7717 
Lunch Eligible 
White, Non-Hispanic 5617 4883 5617 4567 5283 7283 
Other 20633 25683 24867 25233 23983 26967 
paced ae 2683 2767 2550 2567 2283 2750 
Language Proficienc 
aie (ELL) y 2700 2467 2267 1783 1900 1667 


Classification Accuracy Results 


Results of our classification accuracy analyses are presented for fall (Table 2), Winter 


(Table 3), and Spring (Table 4). 


Table 2 
Classification Accuracy: Fall eassCBM MCRC Predicting SBAS ELA Performance 
Grade 3M qn 5m : 6 7m ga 
Criterion SBAS SBAS SBAS SBAS SBAS SBAS 
English English English English English English 
Language | Language Language | Language Language Language 
Arts Arts Arts Arts Arts Arts 
Cut points 40" 40" 40" 40" 40" 40" 
percentile | percentile | percentile | percentile | percentile | percentile 
False Positive Rate 0.19 0.14 0.18 0.16 0.21 0.24 
False Negative Rate 0.34 0.34 0.30 0.37 0.31 0.26 
Sensitivity 0.62 0.55 0.62 0.55 0.57 0.62 
Specificity 0.83 0.91 0.87 0.88 0.86 0.85 
Positive Predictive Power 0.81 0.86 0.82 0.84 0.79 0.76 
Te cenye PICCENE 0.66 0.66 0.70 0.63 0.69 0.74 
Power 
Oye ae ase ean 0.72 0.72 0.75 0.70 0.72 0.75 
Rate 
Area Under the Curve 
(AUC) 0.81 0.84 0.83 0.79 0.80 0.80 
AUC Estimate’s 95% 
Confidence Interval: 0.79 0.82 0.81 0.76 0.78 0.78 
Lower Bound 
AUC Estimate’s 95% 
Confidence Interval: 0.83 0.86 0.85 0.81 0.82 0.82 
Upper Bound 
aaa 5 
ee ae eae 0.57 0.61 0.50 0.37 0.48 0.36 
Sensitivity 
eas A 
SPEC ene aes 0.69 0.72 0.68 0.58 0.64 0.57 
Sensitivity 
peat a 
Se een 0.77 0.80 0.80 0.77 0.77 0.73 
Sensitivity 
Table 3 
Classification Accuracy: Winter easv,CBM MCRC Predicting SBAS ELA Performance 
Grade 3rd qn 5h 62 7h gih 
SBAS SBAS SBAS SBAS SBAS SBAS 
sia be English English English English English English 
Criterion 
Language Language Language Language Language Language 
Arts Arts Arts Arts Arts Arts 
408 408 408 40" 40" 40" 
Cut points ; : : F P P 
percentile percentile percentile percentile percentile percentile 
False Positive Rate 0.16 0.17 0.19 0.18 0.22 0.30 


False Negative Rate 0.34 0.34 0.25 0.44 0.34 0.28 
Sensitivity 0.61 0.57 0.72 0.40 0.53 0.63 
Specificity 0.87 0.88 0.84 0.90 0.86 0.78 
Posie Prepichine 0.84 0.83 0.81 0.82 0.78 0.70 
Power 
Nee ves tecicuye 0.66 0.66 0.75 0.56 0.66 0.72 
Power 
Oe 0.73 0.72 0.78 0.63 0.70 0.71 
Classification Rate ; ; ; ; ; ; 
Area Under the 
0.83 0.82 0.84 0.75 0.78 0.76 
Curve (AUC) 
AUC Estimate’s 
0 
Bos Meee 0.81 0.79 0.82 0.72 0.75 0.74 
Interval: Lower 
Bound 
AUC Estimate’s 
0 
Heh eek weinadad 0.85 0.84 0.86 0.77 0.80 0.79 
Interval: Upper 
Bound 
Specificity Value at 
90% Sensitivity 0.45 0.57 0.53 0.36 0.37 0.34 
Spey Vale Dt 0.77 0.69 0.72 0.55 0.59 0.50 
80% Sensitivity 
ppes ney Vale 0.77 0.78 0.84 0.71 0.75 0.64 
70% Sensitivity 
Table 4 
Classification Accuracy. Spring easyCBM MCRC Predicting SBAS ELA Performance 
Grade 3rd qn 5h oe zi gh 
Criterion SBAS SBAS SBAS SBAS SBAS SBAS 
English English English English English English 
Language Language Language Language Language Language 
Arts Arts Arts Arts Arts Arts 
Cut points 40" 40" 40" 40" 40" 40" 
percentile percentile percentile percentile percentile percentile 
False Positive Rate 0.10 0.19 0.26 0.14 0.26 0.28 
False Negative Rate 0.36 0.33 0.24 0.36 0.25 0.24 
Sensitivity 0.55 0.56 0.67 0.49 0.70 0.69 
Specificity 0.93 0.87 0.81 0.92 0.78 0.79 
Poste TiS deiNe 0.90 0.81 0.74 0.86 0.74 0.72 
Power 
DIRep nye RICE S 0.64 0.67 0.76 0.64 0.75 0.76 
Power 
Abbas 0.73 0.72 0.75 0.70 0.74 0.74 
Classification Rate : ; ; : j : 
Area Under the 
0.85 0.80 0.82 0.80 0.81 0.80 
Curve (AUC) 
AUC Estimate’s 
0 
gee onsen 0.83 0.77 0.79 0.78 0.79 0.78 


Interval: Lower 
Bound 


AUC Estimate’s 
95% Confidence 
Interval: Upper 
Bound 

Specificity Value at 
90% Sensitivity 
Specificity Value at 
80% Sensitivity 
Specificity Value at 
70% Sensitivity 


0.87 0.82 0.84 0.82 0.83 0.83 


0.10 0.19 0.26 0.14 0.26 0.28 


0.36 0.33 0.24 0.36 0.25 0.24 


0.55 0.56 0.67 0.49 0.70 0.69 


Reliability Methods 


Split-half reliability and Cronbach’s Alpha are both estimates of the internal consistency 
of the MCRC measures. Because the easy CBM MCRC measures are often administered for a set 
period of time (typically 30-45 minutes), not all students will complete all items. Having an 
internally-consistent measure, where scores on two split halves of the assessment are correlated 
with one another, provides some reassurance that scores obtained when students complete only 
some of the items (for instance, when they “time out” after responding to only half of the 
possible items on the assessment) reflect the distribution of scores that would be obtained were 
the entire test completed. Prior to analysis, students who had not responded to any items on a 
particular MCRC measure were removed from the dataset. The measures were analyzed for 
internal consistency using Cronbach’s Alpha and Split-half reliability (first half/second half). For 
the split-half reliability, the measures were analyzed using the first half to the median compared 
to the second half. 

Sample and Setting: Reliability Analyses 

Demographic information for the convenience sample used for both the Split-half and 

Cronbach’s Alpha analyses are presented below. The study was conducted using values from the 


fall and winter 2013-2014 Vocabulary benchmark assessments. The fall benchmark was taken by 


20,252 grade 2 students; 23,694 grade 3 students; 17,850 grade 4 students; 26,978 grade 5 
students; 17,222 grade 6 students; 12,798 grade 7 students; and 8,965 grade 8 students. The 
winter benchmark was taken by 19,158 grade 2 students; 21,807 grade 3 students; 15,031 grade 4 
students; 23,146 grade 5 students; 15,575 grade 6 students; 11,506 grade 7 students; and 9,464 
grade 8 students. Students of American Indian or Alaskan Native descent comprised 1-4% of the 
sample, and Asian students made up 2-3% of the sample across all grades. Black or African 
American students made up 10-19% of the sample in grade 2 and 3-5% of the sample in grade 3- 
8. Native Hawaiian or other Pacific Islander students made up 0-1% and students identified as 
Two or more Races constituted 1-2% of the sample across all grades. Lastly, White students 
made up 44-55% of the sample, and those classified as Unknown ethnicity made up 29-47% of 
the sample across all grades. Similarly, students identified as Hispanic/Latino made up 8-16% 
of the sample and students identified as Not Hispanic/Latino made up 48-70% of the sample, 
varying by grade level. The percentage of ELL students in the sample had a range of 16-33%. 
Students identified by their districts as disabled constituted 17-31% of the sample. Males made 


up 49-53% of the sample. 


Reliability Results 

Table 5 
Reliability Results 

Type of 95% Confidence 95% Confidence 

YPe © Grade n Coefficient Interval*: Lower Interval*: Upper 
Reliability 
Bound Bound 

Cronbach's 2 23461 0.75 0.75 0.75 
Cronbach's 3 25074 0.73 0.73 0.74 
Cronbach's 4 20681 0.79 0.79 0.79 
Cronbach's 5 30663 0.78 0.78 0.79 
Cronbach's 6 18135 0.71 0.70 0.71 
Cronbach's 7 15297 0.73 0.73 0.74 


Table 5 


Reliability Results 
Tgpeek 95% Confidence 95% Confidence 
sane Grade n Coefficient Interval*: Lower Interval*: Upper 
Reliability 
Bound Bound 

Cronbach's 8 17639 0.66 0.66 0.66 

Split-half 2 23461 0.72 0.75 0.77 

Split-half 3 25074 0.67 0.74 0.76 

Split-half 4 20681 0.76 0.79 0.81 

Split-half 5 | 30663 0.76 0.80 0.82 

Split-half 6 18135 0.65 0.72 0.74 

Split-half 7 | 15297 0.69 0.75 0.77 

Split-half 8 17639 0.60 0.68 0.71 
Table 6 
Reliability Results, by Subgroup 

are 95% Confidence 
Type of Reliability Subgroup Grade n Coefficient latenak Interval: Upper 
Lower Bound Bowne 

Cronbach's alpha (GenEd 2 15829 0.85 0.85 0.86 
Cronbach's alpha |SPED 2 1766 0.81 0.8 0.82 
Cronbach's alpha (GenEd 3 18797 0.83 0.83 0.84 
Cronbach's alpha |SPED 3 1918 0.87 0.86 0.88 
Cronbach's alpha (GenEd 4 13330 0.82 0.81 0.82 
Cronbach's alpha |SPED 4 1988 0.85 0.84 0.86 
Cronbach's alpha (GenEd 5 19859 0.77 0.77 0.77 
Cronbach's alpha |SPED 5 3440 0.84 0.83 0.84 
Cronbach's alpha (GenEd 6 13086 0.77 0.77 0.78 
Cronbach's alpha |SPED 6 2487 0.87 0.86 0.87 
Cronbach's alpha (GenEd 7 9600 0.77 0.76 0.77 
Cronbach's alpha |SPED 7 1955 0.82 0.81 0.83 
Cronbach's alpha (GenEd 8 7534 0.73 0.73 0.74 
Cronbach's alpha |SPED 8 1931 0.84 0.83 0.84 
Split-half GenEd 2 1829 0.84 0.85 0.86 
Split-half SPED 2 1766 0.79 0.81 0.83 
Split-half GenEd 3 1797 0.83 0.85 0.86 
Split-half SPED 3 1918 0.86 0.87 0.88 
Split-half GenEd 4 1330 0.81 0.84 0.85 
Split-half SPED 4 1988 0.83 0.86 0.87 
Split-half GenEd 5 1859 0.78 0.8 0.81 
Split-half SPED 5 3440 0.82 0.84 0.85 


Table 6 


Reliability Results, by Subgroup 


95% 


races 


Cisnpacies 95% Confidence 
Type of Reliability Subgroup Grade n Coefficient eae Interval: Upper 
Lower Bound Bound 
Split-half GenEd 6 1086 0.78 0.81 0.82 
Split-half SPED 6 2487 0.85 0.87 0.89 
Split-half GenEd 7 9600 0.77 0.79 0.81 
Split-half SPED 7 1955 0.79 0.82 0.84 
Split-half GenEd 8 7534 0.74 0.77 0.78 
Split-half SPED 8 10931 0.82 0.85 0.86 
Cronbach's alpha [F 2 9689 0.85 0.85 0.86 
Cronbach's alpha |M 2 10176 0.85 0.85 0.86 
Cronbach's alpha |M 3 11293 0.84 0.83 0.84 
Cronbach's alpha [F 3 11958 0.85 0.84 0.85 
Cronbach's alpha |M 4 8465 0.83 0.82 0.83 
Cronbach's alpha [F 4 8975 0.84 0.84 0.85 
Cronbach's alpha iM 5 12958 0.79 0.78 0.79 
Cronbach's alpha [F 5 13550 0.81 0.8 0.81 
Cronbach's alpha [F 6 7967 0.79 0.78 0.8 
Cronbach's alpha |M 6 8826 0.83 0.82 0.83 
Cronbach's alpha |M 7 5819 0.79 0.79 0.8 
Cronbach's alpha |F 7 6658 0.82 0.81 0.83 
Cronbach's alpha |F 8 4969 0.75 0.74 0.75 
Cronbach's alpha |M 8 4193 0.81 0.8 0.81 
Split-half F 2 9689 0.84 0.85 0.86 
Split-half M 2 10176 0.84 0.85 0.86 
Split-half M 3 11293 0.83 0.85 0.86 
Split-half F 3 11958 0.84 0.86 0.87 
Split-half M 4 8465 0.82 0.84 0.85 
Split-half F 4 8975 0.83 0.86 0.87 
Split-half M 5 12958 0.79 0.81 0.82 
Split-half F 5 13550 0.81 0.83 0.84 
Split-half F 6 7967 0.8 0.83 0.84 
Split-half M 6 8826 0.82 0.85 0.86 
Split-half M 7 5819 0.79 0.81 0.83 
Split-half F 7 6658 0.81 0.84 0.85 
Split-half F 8 4969 0.75 0.78 0.79 
Split-half M 8 4193 0.81 0.83 0.84 
Cronbach's alpha White 2 10999 0.85 0.85 0.86 
Gronbachsaiphac | oor me 2 | 1434 0.86 0.85 0.87 


Table 6 


Reliability Results, by Subgroup 


95% 


Islander 


Cinpactes 95% Confidence 
Type of Reliability Subgroup Grade n Coefficient ineeal: Interval: Upper 
Lower Bound Bound 

Cronbach's alpha [Asian 2 631 0.86 0.84 0.87 

Cronbach's alpha eas a Rican 2 2254 0.82 0.80 0.83 
American 
American Indian 

Cronbach's alpha jor Alaskan 2 826 0.83 0.81 0.85 
Native 
Native Hawaiian 

Cronbach's alpha jor Other Pacific 2 115 0.82 0.77 0.87 
Islander 

Cronbach's alpha White 3 13963 0.83 0.83 0.84 

Cronbach's alpha Lage pltican 3 2409 0.85 0.84 0.86 
American 

Cronbach's alpha [Asian 3 636 0.83 0.82 0.85 
Native Hawaiian 

Cronbach's alpha jor Other Pacific 3 127 0.83 0.78 0.87 
Islander 

Cronbach's alpha [' or More ae ||| a52a 0.84 0.83 0.85 
races 
American Indian 

Cronbach's alpha jor Alaskan 3 1138 0.83 0.82 0.85 
Native 

Cronbach's alpha White 4 9351 0.83 0.82 0.83 

Cronbach's alpha age aiuiean 4 2334 0.81 0.80 0.82 
American 

Cronbach's alpha [Asian 4 590 0.83 0.81 0.85 
American Indian 

Cronbach's alpha jor Alaskan 4 533 0.80 0.78 0.83 
Native 
Native Hawaiian 

Cronbach's alpha jor Other Pacific 4 115 0.83 0.79 0.88 
Islander 

Cronbach's alpha ['Y or More 4 | 1394 0.82 0.81 0.84 
races 

Cronbach's alpha White 5 13551 0.78 0.78 0.79 

Cronbach's alpha’ Pan eA"), E. | ages 0.79 0.78 0.80 
American 

Cronbach's alpha [Asian 5 713 0.83 0.81 0.85 
Native Hawaiian 

Cronbach's alpha jor Other Pacific 5 160 0.72 0.66 0.79 


Table 6 
Reliability Results, by Subgroup 
0 
en 95% Confidence 
Type of Reliability Subgroup Grade n Coefficient ear Interval: Upper 
Lower Bound Round 

Cronbach's alpha |W oF More 5 | 1858 0.75 0.74 0.77 
races 
American Indian 

Cronbach's alpha jor Alaskan 5 544 0.78 0.75 0.81 
Native 

Cronbach's alpha White 6 8556 0.80 0.80 0.81 

Cronbach's alpha [Asian 6 521 0.84 0.82 0.86 

Cronbach's alpha ['Y or More 6 | 1217 0.78 0.76 0.80 
races 

Cronbach's alpha Hees AMA 6 2571 0.81 0.80 0.82 
American 
Native Hawaiian 

Cronbach's alpha jor Other Pacific 6 130 0.78 0.73 0.84 
Islander 
American Indian 

Cronbach's alpha jor Alaskan 6 350 0.78 0.74 0.81 
Native 

Cronbach's alpha White 7 5961 0.80 0.79 0.81 

Cronbach's alpha |W or More 7 | 980 0.78 0.76 0.80 
races 

Cronbach's alpha ees a pan 7 2556 0.80 0.79 0.82 
American 

Cronbach's alpha [Asian 7 281 0.78 0.75 0.82 
American Indian 

Cronbach's alpha jor Alaskan 7 327 0.76 0.73 0.80 
Native 
Native Hawaiian 

Cronbach's alpha jor Other Pacific 7 43 0.81 0.73 0.89 
Islander 

Cronbach's alpha PlackorAfrican) 5 | g7¢q4 | 0.79 0.79 0.80 
American 

Cronbach's alpha White 8 54052 0.77 0.77 0.77 

Cronbach's alpha Nae dia 8 | 7326]; 0.77 0.76 0.78 

Cronbach's alpha [Asian 8 3720 0.79 0.78 0.80 
Native Hawaiian 

Cronbach's alpha jor Other Pacific 8 544 0.75 0.72 0.78 


Islander 


Table 6 


Reliability Results, by Subgroup 


0 
en 95% Confidence 
Type of Reliability Subgroup Grade n Coefficient ae Interval: Upper 
Lower Bound Bound 

American Indian 

Cronbach's alpha jor Alaskan 8 2296 0.79 0.77 0.80 
Native 

Split-half White 2 10999 0.84 0.85 0.86 

Split-half eee 2 | 1434) 0.84 0.86 0.87 
races 

Split-half Asian 2 631 0.83 0.86 0.88 

Split-half poe cmonee sl) “2; |l|oaear||| G80 0.82 0.83 
[American 
American Indian 

Split-half lor Alaskan 2 826 0.81 0.83 0.85 
Native 
Native Hawaiian 

Split-half lor Other Pacific 2 115 0.76 0.82 0.87 
Islander 

Split-half White 3 13963 0.83 0.85 0.86 

Split-half Po eee) 2g ||[SyaRe! |) 30383 0.85 0.86 
[American 

Split-half Asian 3 636 0.83 0.85 0.87 
Native Hawaiian 

Split-half lor Other Pacific 3 127 0.78 0.85 0.89 
Islander 

Split-half Pee 3 | 1521) 0.83 0.85 0.87 
races 
American Indian 

Split-half lor Alaskan 3 1138 0.82 0.84 0.86 
Native 

Split-half White 4 9351 0.82 0.85 0.86 

Split-half Bere, SUES” san | oaaa:| 20979 0.82 0.83 
[American 

Split-half Asian 4 590 0.82 0.84 0.87 
American Indian 

Split-half ior Alaskan 4 533 0.78 0.82 0.84 
Native 
Native Hawaiian 

Split-half lor Other Pacific 4 115 0.78 0.84 0.89 
Islander 

Split-half eee 4 | 1394] 0.81 0.84 0.86 
races 

Split-half White 5 13551 0.79 0.81 0.83 


Table 6 
Reliability Results, by Subgroup 
0 
en 95% Confidence 
Type of Reliability Subgroup Grade n Coefficient ieee Interval: Upper 
Lower Bound Bound 

Split-half Peer oMeem| < <6: toad || “O97 0.80 0.82 
[American 

Split-half Asian 5 713 0.82 0.85 0.88 
Native Hawaiian 

Split-half lor Other Pacific 5 160 0.67 0.74 0.80 
Islander 

Split-half ee ae 5 | 1858| 0.76 0.79 0.81 
races 
American Indian 

Split-half lor Alaskan 5 544 0.76 0.80 0.83 
Native 

Split-half White 6 8556 0.81 0.84 0.85 

Split-half Asian 6 521 0.83 0.87 0.89 

Split-half Pees 6 |1217| 0.78 0.82 0.84 
races 

Split-half creme) See Ve gtll) G80 0.83 0.85 
[American 
Native Hawaiian 

Split-half lor Other Pacific 6 130 0.74 0.81 0.86 
Islander 
[American Indian 

Split-half ior Alaskan 6 350 0.76 0.81 0.84 
Native 

Split-half White 7 5961 0.79 0.82 0.83 

Split-half Pee nee 7 | 980 0.78 0.81 0.83 
races 

Split-half Poe aes ee) ‘age tlleoemee || says 0.82 0.83 
[American 

Split-half Asian 7 281 0.75 0.80 0.84 
American Indian 

Split-half or Alaskan 7 327 0.73 0.78 0.82 
Native 
Native Hawaiian 

Split-half lor Other Pacific 7 43 0.72 0.82 0.89 
Islander 

Split-half peer | . dae ||l-Gaae|| soe 0.81 0.83 
[American 

Split-half White 8 5052 0.78 0.80 0.81 

Split-half ee eens 8 | 7326 0.78 0.80 0.81 


races 


Table 6 
Reliability Results, by Subgroup 
0 
en 95% Confidence 
Type of Reliability Subgroup Grade n Coefficient ineral: Interval: Upper 
Lower Bound Bound 
Split-half Asian 8 3720 0.79 0.82 0.83 
Native Hawaiian 
Split-half lor Other Pacific 8 544 0.74 0.78 0.81 
Islander 
American Indian 
Split-half lor Alaskan 8 2296 0.78 0.81 0.82 
Native 


Discussion: Reliability 
The results of the test-retest and alternate-form reliability analyses suggested acceptable 


form equivalence. 


Validity Methods 

We used the Smarter Balanced English Language Arts Assessment as our criterion 
measure. This measure is completely independent from the screening measure. SBAS is a large- 
scale assessment in wide use across the United States as a state accountability measure. Because 
it is used by so many states for their accountability measure, school districts are quite interested 
in the relation between SBAS and easyCBM MCRC. 
Setting and Sample: Validity Study 

Data for this study came from a convenience sample provided by two school districts in 
the Pacific Northwest. All students enrolled in school and present during the three-week 
easyCBM Benchmark Assessment windows in the fall (September 2014), winter (January 2015) 
and spring (May 2015) were administered the easyCBM assessments. All enrolled students were 


likewise administered the Smarter Balanced assessments during the testing window provided by 


the state in the spring of 2015. The data set provided by the districts included easy CBM CCSS 
Math, Passage Reading Fluency, Vocabulary, and Multiple Choice Reading Comprehension 
(MCRC) as well as Smarter Balanced Math and English Language Arts total scores for students 
enrolled in grades 3-8. District 1 provided data for Grades 3-8, while District 2 provided data for 
Grades 4-8. In addition, District 1 provided demographic information, while District 2 
(approximately '4 the size of the first district) did not. Known demographics of the sample are 
provided in Table 7. Because of the missing demographics from a large proportion of the sample, 
the percentages for each of the demographic variables are calculated based on the students in the 


sample whose data included full-resolution demographic information. 


Table 7 
Sample Demographics 


Missing 
Demographic Female Hispanic SpEd ELL 

Grade Data 

# % # % # % # % # % 
3 33 3 492 48 187 18 87 8 67 7 
4 328 24 523 50 217 21 100 10 62 6 
) 295 23 483 48 159 16 89 9 39 4 
6 291 22 505 49 180 17 95 9 27 3 
7 280 23 456 48 185 19 78 8 29 3 
8 266 20 526 50 192 18 83 8 22 2 


During data cleaning, data from students who were administered the Alternate Assessment rather 
than the General Education assessment were removed from the dataset prior to further analyses. 
In all, six students each from Grades 4, 6, and 7 and three students from Grade 5 were removed 


from the dataset in this step. Data from all additional students were retained. 


Validity Analyses 


We analyzed the data using bivariate correlations and linear regression using SPSS 


software. 
Table 8 
Criterion-Related Validity Evidence 
95% 95% 
ey ae : fid Confidence 
Lower Bound| Upper Bound 
Predictive (Fall easyCBM oe 
Tedaictive all easy’ 
MCRC > Spring SBAS) 3 Balanced 1239 0.62 0.58 
Assessment 
Predictive (Win. easy CBM Sta 
MCRC > Spring SBAS) 3 Balanced 1285 0.63 0.59 
Assessment 
Cc t(S CBM Smarter 
oncurrent (Spr easy! 
MCRC > Spring SBAS) 3 Balanced 1288 0.68 0.64 
Assessment 
Smarter 
Concurrent (Spr easy CBM 
MCRC > Spring SBAS) 4 Balanced 1443 0.68 0.65 
Assessment 
Concurrent (Spr easyCBM evades 
MCRC > Spring SBAS) 4 Balanced 1483 0.64 0.60 
Assessment 
Cc t(S CBM Smarter 
oncurrent (Spr easy' 
MCRC > Spring SBAS) 4 Balanced 1307 0.62 0.58 
Assessment 
c t (Spr easyCBM Sian 
oncurren pr easy 
MCRC > Spring SBAS) 5 Balanced 1531 0.67 0.63 
Assessment 
Cc t(S CBM Smarter 
oncurrent (Spr easy! 
MCRC > Spring SBAS) 5 Balanced 1570 0.68 0.64 
Assessment 
Cc t(S CBM Smarter 
oncurren pr easy 
MCRC > Spring SBAS) 5 Balanced 1333 0.64 0.60 
Assessment 
Concurrent (Spr easyCBM Smarice 
MCRC > Spring SBAS) 6 Balanced 1505 0.60 0.55 
Assessment 
Cc t(S CBM Smarter 
oncurrent (Spr easy! 
MCRC > Spring SBAS) 6 Balanced 1554 0.54 0.50 
Assessment 
Cc t(S CBM Smarter 
oncurren pr easy 
MCRC > Spring SBAS) 6 Balanced 1297 0.65 0.61 
Assessment 
Concurrent (Spr easy CBM Smarter 
Ninbies e ona 7 1433 0.65 0.61 


Table 8 


Criterion-Related Validity Evidence 


95% 95% 
Type of Validity Grade Criterion n Coefficient Palatal ener 
Lower Bound| Upper Bound 
SOUP eye q ce 7 1476 0.59 0.55 
MCRC > Spring SBAS) hceotgeal . , 
Onsurre nk (SpE casy CPM q See 7 1200 0.62 0.58 
MCRC > Spring SBAS) Resdeenient ; : 
COE Oe EN | Se ae 8 1475 0.63 0.59 
MCRC > Spring SBAS) TORE / ; 
Coneurre ups Sao 8 nes 8 1535 0.58 0.54 
MCRC > Spring SBAS) PROP : ; 
Roneur eab(Spr easyCBM 8 ae 8 1250 0.64 0.60 
MCRC > Spring SBAS) Nees ceiient . , 
Discussion: Validity Evidence 


Data from these validity studies support the concurrent and predictive validity of the tool. 


Correlations between the easy CBM MCRC measures and an external measure of English 


Language Arts that includes reading comprehension as a tested construct suggest that the 


easyCBM MCRC assessments are, indeed, capturing important information about students’ 


ability to make sense of what they are reading. The easy CBM MCRC measures consistently 


predict student performance on other measures of English Language Arts. 


